The present disclosure relates to an image capture device, an image capture device control method, and a program. In particular, the present disclosure relates to an image capture device, an image capture device control method, and a program that generate an image with a dynamic range that is wide.
A solid image capture element that is used in a video camera or a digital still camera, such as a CCD image sensor or a complementary metal oxide semiconductor (CMOS) image sensor, accumulates an electrical charge that corresponds to the amount of incident light and performs a photoelectric conversion that outputs an electrical signal that corresponds to the accumulated electrical charge. However, there is an upper limit to the amount of the electrical charge that is accumulated in the photoelectric conversion element, and when more than a fixed amount of light is received, the amount of the accumulated electrical charge reaches a saturation level, such that what are called blown out highlights, which are set at a saturated brightness level, occur in regions of the photographic subject where the brightness is greater than a fixed value.
In order to prevent this sort of phenomenon from occurring, processing is performed that adjusts the exposure time by controlling the period during which the electrical charge is accumulated in the photoelectric conversion element, in accordance with variations in the outside light and the like, and that also adjusts the sensitivity to an optimum value. For example, for a bright subject, the exposure time is shortened by using a faster shutter speed, shortening the period during which the electrical charge is accumulated in the photoelectric conversion element, and the electrical signal is output before the amount of the accumulated electrical charge reaches the saturation level. This sort of processing makes it possible to output an image that accurately reproduces the gray-scale levels of the subject.
However, in capturing an image of a subject that has a mixture of bright regions and dark regions, using a fast shutter speed means that the exposure time will not be sufficient for the dark portions, so the signal-to-noise ratio worsens, and the image quality deteriorates. In order to accurately reproduce the brightness levels of the bright portions and the dark portions in a captured image of a subject that has a mixture of bright regions and dark regions, processing must be performed that increases the exposure time and achieves a high signal-to-noise ratio for the image sensor pixels where the amount of the incident light is low and that avoids saturation in the pixels where the amount of the incident light is high.
As a technique for implementing this sort of processing, a technique is known that sequentially captures and combines a plurality of images with different exposure times. Specifically, an image with a long exposure time and an image with a short exposure time are captured separately in sequence. The technique generates a single image by performing combining processing that uses the long exposure time image for the dark image regions and uses the short exposure time image for the bright image regions where the highlights are blown out in the long exposure time image. Combining a plurality of images with different exposures in this manner makes it possible produce an image with a dynamic range that is wide and in which there are no blown out highlights, that is, a wide dynamic range image (a high dynamic range (HDR) image).
For example, Japanese Patent Application Publication No. JP-A 2008-99158 discloses a configuration that produces a wide dynamic range image by combining a plurality of images with different amounts of exposure. The processing will be explained with reference to FIG. 1. An image capture element, in capturing moving images, for example, outputs image data for two different exposure times within a video rate (30 to 60 fps). In capturing still images, too, the image data are generated for two different exposure times and output. FIG. 1 is a figure that explains characteristics of images (a long exposure time image, a short exposure time image) that the image capture element generates and that have two different exposure times. The horizontal axis is time (t), and the vertical axis is an accumulated electrical charge (e) in a light-receiving photo diode (PD) that configures a photoelectric conversion element that corresponds to one pixel of a solid image capture element.
For example, in a case where the amount of light that the light-receiving photo diode (PD) is large, that is, where it corresponds to a bright subject, the accumulated electrical charge increases rapidly as time elapses, as shown in a high brightness region 11 that is shown in FIG. 1. In contrast, in a case where the amount of light that the light-receiving photo diode (PD) is small, that is, where it corresponds to a dark subject, the accumulated electrical charge increases slowly as time elapses, as shown in a low brightness region 12 that is shown in FIG. 1.
The time from t0 to t3 is equivalent to an exposure time TL for acquiring the long exposure time image. The line that is shown in the low brightness region 12 shows that the accumulated electrical charge at the time t3, even as the long exposure time TL, has not reached the saturation level (unsaturated point Py), and an accurate gray-scale expression can be produced according to the gray level of the pixel that is set using an electrical signal that is produced based on the accumulated electrical charge (Sa).
However, the line that is shown in the high brightness region 11 clearly indicates that the accumulated electrical charge has already reached the saturation level (saturated point Px) before it reaches the time t3. Therefore, in the high brightness region 11, only a pixel value that corresponds to an electrical signal at the saturation level is produced from the long exposure time image, resulting in a pixel that is blown out.
Accordingly, in the high brightness region 11, the accumulated electrical charge is swept out of the light-receiving photo diode (PD) once before the time t3 is reached, for example, at a time t1 (a charge sweeping starting point P1) that is shown in FIG. 1. The charge sweeping does not sweep out the entire accumulated electrical charge in the light-receiving photo diode (PD), but sweeps it down to an intermediate voltage hold level that is controlled by the photo diode (PD). After the charge sweeping processing, the light-receiving photo diode (PD) is once again exposed to light for a short time that is defined as an exposure time TS (from t2 to t3). That is, a short time exposure is made for the period from a short exposure time starting point P2 to a short exposure time ending point P3, which are both shown in FIG. 1. An accumulated electrical charge (Sb) is produced by the short time exposure, and the gray level of the pixel is set using an electrical signal that is produced based on the accumulated electrical charge (Sb).
Note that in the setting of the pixel value using the electrical signal that is based on the accumulated electrical charge (Sa) that is produced by the long time exposure in the low brightness region 12 and using the electrical signal that is based on the accumulated electrical charge (Sb) that is produced by the short time exposure in the high brightness region 11, an estimated accumulated electrical charge is computed for a case in which the exposure times are the same in the two regions, an electrical signal output value that corresponds to the estimated accumulated electrical charge is computed, and the pixel value is set based on the results of the computations.
Combining the short exposure time image and the long exposure time image in this manner makes it possible to produce an image that has no blown out highlights and a dynamic range that is wide.
In addition, for an image capture device that captures a wide dynamic range image, a method for performing exposure control for individual pixels based on advance image capture information, a method for combining images while varying the exposure spatially, and the like are known.
For example, Japanese Patent Application Publication No. JP-A 2001-358989 and Japanese Patent Application Publication No. JP-A 2010-136205 disclose configurations that control exposure and achieve wide dynamic range image capture that maintains high resolution by storing exposure control information based on brightness information for preliminarily captured images, then incorporating that information as parameters in the conditions for the next image capture.
However, with the method that utilizes a plurality of captured images, as has been explained with reference to FIG. 1, and with the configuration that performs processing that controls the exposure based on the advance image capture information, in a case where the subject is moving, fluctuations occur in the pixel values. Therefore, with the method that utilizes a plurality of captured images, as has been explained with reference to FIG. 1, erroneous pixel values are set when the combined image is generated.
Furthermore, with the method that performs exposure control based on the brightness information for preliminarily captured images, the exposure control information that is set for the individual pixels in advance becomes erroneous information, and it becomes impossible to achieve optimum control.
Problems therefore arise, in that when the pixel that has changed from dark to bright, the exposure is much greater than what was anticipated, so the pixel becomes saturated, and when the pixel that has changed from bright to dark, the exposure is much less than what was anticipated, so the signal-to-noise ratio decreases, for example.
Furthermore, in Japanese Patent Application Publication No. JP-A 2010-110004, a wide dynamic range is achieved by providing a plurality of sensors with different sensitivities, arranging them in a fixed pattern, and interpolating low exposures and high exposures among the individual pixels.
With this method the exposure of each pixel is fixed, irrespective of the time, so the method is resistant to the effects of a moving subject. However, because the low sensitivity and high sensitivity sensors are arranged in a fixed pattern, the sampling sensitivity range is wider than with the known RGB sensors, so a problem occurs in that the image resolution decreases.